Merge main into stable#3604
Conversation
Pickled (https://docs.pickled.dev) runs scripted scenarios across a matrix of interfaces, sources, and toolsets and scores answers with deterministic checks. This config covers one scenario today (the custom React toolbar question) across two interfaces (Claude Code haiku, OpenAI Responses) and four context-delivery paths (none, web, the official SuperDoc Mintlify docs MCP, Context7 MCP). 16 cells per run. Sits alongside evals/ rather than under it: evals/ is the Promptfoo suite that scores the SuperDoc tool surface; this is the outside-in view of how agents talk about SuperDoc when asked to build with it. Run with: bunx @pickled-dev/cli check .
- Added validation for the MCP_PRESET environment variable in `server.ts` to ensure only supported presets are accepted at startup, preventing silent misconfigurations. - Introduced a new preset registry in `presets.ts`, allowing for the management of LLM tool presets, with the initial implementation supporting only the 'legacy' preset. - Updated the Node SDK to expose preset-related functions (`getPreset`, `listPresets`, `DEFAULT_PRESET`) for easier access to preset information. - Created tests for preset validation and registry functionality, ensuring that unknown presets trigger appropriate errors and that the legacy preset behaves as expected. - Added corresponding Python SDK support for the preset registry, mirroring the Node implementation for consistency across languages.
A searchable "Smart tags" palette in the contract-templates sidebar. Clicking a
tag inserts it as an inline content control at the caret, and the inserted field
paints with the SAME token look (--tag-* / .smart-tag) as the palette chip - so
the sidebar tag and the in-editor field read as one object. This is the core
custom-SDT story: turn off built-in chrome, style the painted wrapper, author
fields from your own UI.
Insert path (verified): ui.selection.capture() -> bridge the TextTarget to a
collapsed SelectionTarget -> editor.doc.create.contentControl({ at, content,
tag }) -> ui.contentControls.focus(). Adds a behavior test proving collapsed-
caret insertion works (no API gap) and a demo acceptance test for chip -> field.
- Smart tags get a deliberate amber identity (one --tag-* token set drives both the palette chip and the painted in-editor field, so they look identical). - Two-way loop: clicking a smart-field token in the document highlights its sidebar chip (content-control:click); cleared on blur (active-change). - README reframed around the custom content-control UI story (chrome:'none' + host-owned field look + smart-tags authoring), with the new flow documented. - Adds a demo test for the click-token -> highlight-chip sync.
β¦field chip - Remove the floating field chip (sd-field-chip): redundant now fields are styled inline, and it clashed with the amber palette. Drops field-chip.ts and the chip-anchor test (the chip was the only getRect/viewport.observe consumer). - Inline and block fields now share one amber token language: inline as a token pill, block clauses as a quiet left-rail card (a region, not a token). - Kill the jitter: under chrome:'none' SuperDoc resets the SDT border/fill on hover (:hover / .sdt-group-hover) and select (.ProseMirror-selectednode) so consumers own the look; without re-asserting, the box shifted ~2px and lost the amber. We re-assert both states for inline and block to hold the exact box and keep a controlled amber fill. The !important is ours, to win over the reset without coupling to SuperDoc's selector specificity -- a custom-UI styling rough edge (no first-class per-control hook yet) worth a follow-up. - Size the sidebar chips to match the in-editor pills. - Add regression tests asserting the inline pill and block clause boxes stay constant across hover/select (no jitter).
feat: add layered style export
β¦-20260529-215717
β¦SDTs
Reframe the contract-templates demo as a building-block library on a locked
template surface, driven entirely through the public superdoc/ui + editor.doc.*
API with chrome:'none'. This shows the legal-tech workflow: assemble a contract
from governed, reusable Word content controls whose variables stay consistent.
- Enable the formatting toolbar and center the editor. Fold Clauses into a
Template tab; the sidebar is now Template (build) + Values (fill).
- Template tab is a catalog: smart-field chips and clause cards (each with
category / jurisdiction / version and a "used N times" count, plus a
library-only Indemnification clause). Drag or click to insert; a field goes
inline at the caret, a clause snaps to a block boundary. Inserts resolve the
drop point with ui.viewport.positionAt.
- Every control is contentLocked, so it can't be edited by typing. Fields show
their name token (e.g. DISCLOSING_PARTY) as a placeholder. Values are filled
only through the Values form, which broadcasts to every occurrence - including
ones nested in a locked clause (the write briefly unlocks clauses, since a
clause's content lock otherwise silently vetoes nested writes).
- Clauses are assembled from structured parts (prose + {field} slots): inserting
one wraps each slot as a nested, locked inline smart field, so an inserted
Permitted Use carries real Receiving party / Purpose fields like the seeded one.
- Remove the clause version review/replace lifecycle (out of scope here; it's a
separate clause-lifecycle demo). Drop the floating field chip earlier in the arc.
- Rewrite the README and file header to the library model; add tests for locking,
nested-clause broadcast, clause insert, and inserted-clause field nesting.
Three review findings from PR 3541:
1. Restore structured ToolCatalog.tools type. The refactor narrowed the
public catalog row to `unknown[]`, breaking TS consumers that read
tools[i].toolName etc. Move ToolCatalogEntry + ToolCatalogOperation
into presets.ts as public types and tighten the catalog signature.
2. Fail fast on malformed provider bundles. Node and Python preset
loaders previously coerced a missing or non-array `tools` field to
`[]`, hiding broken codegen output behind a silently empty tool
surface. Restore the pre-presets TOOLS_ASSET_INVALID throw at the
preset boundary.
3. Cross-lang parity for empty-string presets. Python choose_tools
treated `{'preset': ''}` as legacy via `or DEFAULT_PRESET`; Node and
MCP both raise PRESET_NOT_FOUND. Use an explicit None check so
Python matches.
Tests added covering structural catalog access, empty-string preset
fail-fast, and cross-lang parity for the empty-string case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Make the clause library a single-use inclusion checklist instead of a duplicate stamp tool. A clause is either "In contract" or available to "Add clause": a clause already placed can't be inserted again - clicking its card reveals the existing section, while an available card adds it (click or drag) and then flips to "In contract". Drops the "used N times" surface. This teaches the right model: fields are reusable variables, clauses are governed sections included once. - Add a library-only "Return of Materials" clause carrying a nested Receiving party slot, so insert-with-nested-fields stays demonstrable now that the seeded Permitted Use is "In contract" and no longer insertable. - Recolor fields and clauses to the SuperDoc brand blue (--sd-color-blue-500/600, per brand.md) instead of amber. They render as tinted/outlined pills, so they stay distinct from the solid-blue primary buttons. - Update tests (single-use status badges, add-once-no-duplicate, nested-field on add), the README, and code comments to the single-use + blue model.
β¦art-tags feat(demo): smart-tags palette for custom SDT fields (SD-3320)
β¦29-215717 π Sync stable β main
fix: make toc toolbar icon configurable
llm tools presets
β¦SD-3322) Under modules.contentControls.chrome:'none' the painter erased the SDT look entirely, so a consumer who wanted a custom field/clause appearance had to target the painted wrapper with !important and reach into internal state classes (.ProseMirror-selectednode, .sdt-group-hover) to keep it stable across hover and selection. That's the wrong "best practice" to teach. Make the chrome-none reset read a --sd-content-controls-custom-* variable layer with default-preserving fallbacks (0-width transparent border, no background / radius / padding). chrome:'none' stays visually empty by default - existing consumers see no change - but a consumer can now paint inline and block controls by setting variables on a data-sdt-* selector. The painter applies them across rest, hover, and selected, so the box stays stable (no jitter) and no !important or state-class selectors are needed. `border` is a full shorthand; block adds a `-border-left` accent rail; background vars cascade (hover from rest, selected from hover). - variables.css: document the custom-* surface; note the built-in chrome still uses the existing --sd-content-controls-* variables. - docs: add a "Style the controls in place" section to the custom-UI content controls guide. - test: assert the surface is wired and default-preserving; existing chrome-none selector + source-order tests are unchanged and still pass (painter-dom 1178/1178).
β¦' (SD-3322) The custom hover background was overridden for LOCKED controls under chrome:'none'. The base lock-hover rules (a built-in tint on inline, transparent on block) have equal specificity to the plain custom hover rules but come later in source order, so they won; the chrome-none lock-hover reset only reset z-index, not background. Re-assert the custom hover background in that reset block - it carries the extra .superdoc-cc-chrome-none class, so it outranks the base lock-hover rules. A locked control now follows --sd-content-controls-custom-*-hover-bg. With no custom var set the default is empty, so the built-in lock-hover tint no longer leaks under chrome:'none' for locked controls (consistently empty). Only the contract-templates demo has locked chrome-none controls, and it wants the custom hover, not the tint. Add a regression test asserting the custom hover vars are re-asserted after the base lock-hover rules (source order = it wins). painter-dom 1179/1179 green.
β¦s (SD-3322) The content-controls theming table themes the built-in chrome. Add a one-line note that under chrome:'none' you style controls with the --sd-content-controls-custom-* variables instead, linking the custom UI guide.
β¦D-3322) Rewrite the contract-templates demo's SDT styling onto SuperDoc's public --sd-content-controls-custom-* variables (from #3590), proving the new API in the real legal-template use case. The demo now styles its inline fields and block clauses with zero !important and zero internal state selectors (.ProseMirror-selectednode, .sdt-group-hover); the painter applies the variables across rest, hover, selected, and locked-hover. This is the copy-pasteable pattern for styling custom SDTs under chrome:'none'. - style.css: replace the per-state !important rules with one variable-setting rule per tag (inline + block); update the host-owned-styling comment. - test: add state coverage - the custom hover background drives a painted field (and wins over the built-in lock-hover tint), the border stays constant across states (no jitter), and no built-in label/chrome leaks. Demo suite 13/13. - docs (Document API > Content controls): correct the contentLocked wording (it rejects Document API content writes too, not just the editor); document the locked-template pattern (unlock -> write -> relock, incl. a locked parent for nested fields); add the single-use governed clause-library pattern alongside versioned reusable sections (kept - it's a valid pattern). - docs (Custom UI > Content controls): add a "Build a custom field system" walkthrough; describe the demo as a full custom contract-template UI. - README: note the demo styles through the public custom variables. Stacked on #3590 (the painter variable layer); retarget to main once it merges.
* refactor: move pm adapter out of layout engine * fix: review comments and build issue * fix: add pm adapter config ref to superdoc config * fix: bring missing file back after merging * fix: move pm layout adapter
β¦ols-chrome-none-css-vars feat(painter-dom): custom SDT styling variables under chrome:'none' (SD-3322)
β¦stom-sdt-vars demo/docs: contract-templates use custom SDT styling variables (SD-3322)
β¦OA field content (SD-3005) (#3538)
β¦nfig chore: add root pickled.yml for agent-legibility checks
β¦-yml chore: remove root pickled.yml
* feat(layout): footnote-aware body pagination (SD-3049/3050/3051)
Make the body paginator demand-aware so footnote-heavy documents pack
body content tight to the separator instead of letting the post-hoc
reserve loop leave visible blank space above the footnote band.
Measured on Harvey NVCA Model SPA (108 footnote refs):
- BEFORE: 57 pages
- AFTER: 53 pages
- Word baseline: 51 pages (within +5%)
Mechanism
---------
PageState gains two fields:
- pageFootnoteReserve : existing per-page reserve, now exposed
to the break decision
- footnoteDemandThisPage : accumulator of measured footnote body
heights for refs anchored on this page
Paragraph layout consults a new optional callback:
- getFootnoteDemandForBlockId(blockId): number
The break decision uses an effective bottom:
additionalDemand = max(0, footnoteDemandThisPage - pageFootnoteReserve)
effectiveBottom = state.contentBottom - additionalDemand
Once the convergence loop has set a correct reserve, additionalDemand is
0 and the new code is a no-op. On pass 1 (no reserve), it provides the
tight-packing signal that prevents the body from filling the page only
to be clawed back by a later reserve relayout.
A safety cap clamps additionalDemand so the page always has room for at
least one body line - otherwise an oversized footnote would drive
effectiveBottom below cursorY and the paginator would advanceColumn
indefinitely.
The per-block demand lookup is built once per layoutDocument call. It
walks the block tree, including table cells (rows[].cells[].blocks /
.paragraph), and resolves each ref's pos to the containing top-level
block. Table-cell refs are attributed to the table block, the unit the
body paginator places on a page.
layout-bridge populates bodyHeightById from measures via
refreshBodyHeights and pre-measures every footnote on every convergence
iteration so migrating refs do not drop from the lookup mid-loop.
Tests
-----
- footnoteBodyDemand.test.ts RED-then-GREEN for block-aware break
+ no-op invariant for non-footnote docs
- footnoteContinuationDemand converged layout reserves carry-forward
demand on the continuation page
- footnoteRefMigration determinism regression: repeated runs
produce identical page counts, reserves,
and ref to page assignments
Refs: SD-2656 SD-3049 SD-3050 SD-3051
Plan: docs/plans/sd-2656-footnote-rendering-fidelity.md
Report: docs/plans/sd-2656-implementation-report.md
* feat(footnote): honor w:numFmt / w:numStart + customMarkFollows (SD-2986 SD-2658)
Inline footnote references and the leading marker inside the footnote
body now honor the OOXML number format / start configured in
w:settings/w:footnotePr. Custom-mark refs (customMarkFollows="1") emit
an empty marker run so the literal symbol in the next OOXML run
renders as the visible mark.
Supported formats: decimal, upperRoman, lowerRoman, upperLetter,
lowerLetter, numberInDash. Unknown formats fall back to decimal.
Single source of truth between the inline ref and the leading marker:
pm-adapter/src/footnote-formatting.ts -> formatFootnoteCardinal()
Used by:
pm-adapter/.../converters/inline-converters/footnote-reference.ts
super-editor/.../layout/FootnotesBuilder.ts
The formatter switch is intentionally inlined (not imported from
@superdoc/layout-engine's formatPageNumber) because pm-adapter sits
upstream of layout-engine in the package graph - see Guard C in
layout-engine/tests/src/architecture-boundaries.test.ts. A drift
detection parity test asserts the two helpers agree on every supported
format for cardinals 1..100:
layout-engine/tests/src/footnote-formatter-parity.test.ts
Settings readers in super-editor/document-api-adapters/document-settings:
readFootnoteNumberFormat(settingsRoot): string | null
readEndnoteNumberFormat(settingsRoot): string | null
readFootnoteNumberStart(settingsRoot): number | null
readEndnoteNumberStart(settingsRoot): number | null
PresentationEditor reads all four up-front and threads the values
through ConverterContext.footnoteNumberFormat / .endnoteNumberFormat
and the per-doc cardinal counter is seeded with the configured start.
customMarkFollows handling preserves pmStart/pmEnd on the empty marker
run so click and selection continue to work at the ref position.
Refs: SD-2656 SD-2986 SD-2986/B1 SD-2986/B2 SD-2658 SD-2662
* docs(footnote): sd-2656 plan + implementation report
End-to-end documentation for the footnote rendering fidelity epic:
docs/superdoc-feature-reports/sd-2656-plan.md
Original implementation plan: ticket inventory across the epic,
OOXML grounding (Β§17.11), code surface map with line numbers,
surgical approach for each slice, RED test scaffolds, falsifiable
success criteria.
docs/superdoc-feature-reports/sd-2656-implementation-report.md
What shipped, with measurements:
- Harvey NVCA: 57 -> 53 pages (Word baseline 51, +5%)
- pnpm test:layout vs superdoc@1.32.0:
535/543 docs (98.5%) byte-identical
5 unique-change docs, all NVCA-style footnote-rich legal
templates (the intended scope)
- pnpm test:visual: "no visual differences found"
- 16,649 unit tests across 5 packages, all green
Slice-by-slice walkthrough (SD-3049 / 3050 / 3051 / 2986/B1+B2 /
2658 / 2662), architecture compliance (Guard C parity test),
pr-reviewer findings + resolutions, deferred work, repro commands.
Refs: SD-2656
* fix(footnote): close review gaps in SD-2656 (demand recharge, endnote numFmt, cache key)
- Re-charge block footnote demand after each advanceColumn so a paragraph
that spills mid-iteration leaves the new page with the right effective
bottom β previously the recharge only fired at iteration top, and a block
that finished its content on the spilled-onto page never charged its
demand there, letting later blocks fill into the footnote band.
- Wire endnoteNumberFormat through endnoteReferenceToBlock and EndnotesBuilder
via the shared formatFootnoteCardinal so documents with w:endnotePr/w:numFmt
render the configured format on both the inline ref and the leading marker.
- Fold numberStart and numberFormat into the FlowBlockCache invalidation
signatures so settings.xml mutations that change numbering format or
starting cardinal evict stale cached reference runs.
- refreshBodyHeights mirrors computeFootnoteLayoutPlan: read measure.height
for image and drawing footnote content so the SD-3049 tight-pack signal
fires for non-text footnotes.
Tests:
- layout-paragraph.test.ts: demand survives advanceColumn within one iteration
- endnote-reference.test.ts: numFmt cases (upperRoman, lowerRoman, fallbacks)
- footnoteBodyDemand.test.ts: tight gap for image-only footnotes
Refs: SD-2656
* fix(footnote): list demand + customMark suppresses body marker (SD-2656)
- refreshBodyHeights now handles list-kind measures (per-item paragraph
line heights + spacingAfter), mirroring buildFootnoteRanges. Without it
list-only footnotes contributed zero demand to the SD-3049 tight-pack
signal and re-introduced the blank body-to-separator gap.
- FootnotesBuilder captures customMarkFollows on the inline ref and skips
the leading marker injection in the footnote body for those ids. Matches
the exporter contract: custom-mark footnotes have no w:footnoteRef in
note content; the literal symbol in the document body is the entire
identification.
Tests:
- footnoteBodyDemand.test.ts: tight gap for a list-only footnote
- FootnotesBuilder.test.ts: customMarkFollows ref does not inject a marker run
* fix(footnote): dedupe block demand by footnote id (SD-2656)
The footnote band already renders each id once per page via
assignFootnotesToColumns. Block-aware body demand must match: when the
same id is referenced multiple times on a page, contribute its body
height once. Previously refByPos kept every occurrence, so two refs to
the same footnote on a page reserved 2Γ the real height and the body
paginator left phantom whitespace above the separator at convergence.
The dedup keeps the first ref position per id (sufficient for the
walker, which only needs to attribute demand to *some* containing
block).
Test: 25 body paragraphs, footnote referenced twice β page 1 must pack
tight with no extra whitespace.
* fix(footnote): charge block demand once, on anchor page (SD-2656)
The block-aware break re-charged blockFootnoteDemand on every page
transition. For a long paragraph that spans pages with a footnote ref
on the first one, continuation pages got the demand subtracted from
their effective body region even though no footnote band renders
there β packing 13β15 lines per page instead of 20 and producing
unnecessary extra pages.
Lock the charge after the first fragment commits. The spill case
(Fix 1, paragraph's first fragment lands after advanceColumn) still
works because re-charging still happens until the first commit; once
the fragment is on the page, the lock prevents continuation pages from
seeing phantom demand.
Test: 50-line paragraph with a single ref on a 20-line-per-page layout
converges to 3 pages (was 4 with per-page recharge).
* fix(footnote): flip separator widths to match ECMA-376 (SD-2985)
Β§17.11.1 w:continuationSeparator β "spans THE WIDTH of the main story's text extents"
Β§17.11.23 w:separator β "spans PART OF the width text extents"
The current code had the two cases inverted: standard separator drawn at full
column, continuation drawn at 30% column. Word renders the opposite.
Test: footnoteSeparatorWidth.test.ts asserts standard β 0.5 Γ contentWidth and
continuation β contentWidth on a fixture that forces footnote spill across pages.
* fix(footnote): customMark refs do not consume an ordinal (SD-2986/SD-2657)
Β§17.11.14 footnoteReference: "shall not increment the numbering for its
associated footnote/endnote numbering format, so that the use of a footnote
with a custom footnote mark does not cause a missing value in the
footnote/endnote values."
The previous numbering walk in PresentationEditor incremented the counter for
every unique footnoteReference id, including those carrying customMarkFollows.
A document with mixed auto + customMark refs and numFmt=upperRoman would
render as I, II, III instead of the spec-mandated I, [custom], II.
Extracted the numbering loop to layout/computeNoteNumbering.ts so the
behavior is directly testable (and shared between footnote + endnote walks
in PresentationEditor). The shared isCustomMarkFollows helper now lives here
too β FootnotesBuilder and EndnotesBuilder will reuse it.
Tests:
- computeNoteNumbering.test.ts (23 cases) β first-appearance numbering,
dedup, custom-mark suppression, OOXML on/off parsing.
* fix(endnote): suppress body marker for customMark refs (parity with footnote)
Β§17.11.14 customMarkFollows applies to both w:footnoteReference and
w:endnoteReference (both extend CT_FtnEdnRef). FootnotesBuilder already skips
the synthetic body marker for custom-mark refs; EndnotesBuilder now mirrors it.
Reuses the shared isCustomMarkFollows helper extracted in the previous commit
(layout/computeNoteNumbering.ts). Removes the local duplicate from
FootnotesBuilder.
Tests:
- EndnotesBuilder.test.ts (4 new cases) β body marker present for normal refs,
suppressed when customMarkFollows is truthy, preserved when "0" / "false".
* feat(footnote): honor section-level w:footnotePr + numRestart=eachSect (SD-2986)
Β§17.11.11 β section-level w:footnotePr overrides document-wide numFmt /
numStart / numRestart. (pos is parsed but ignored per Β§17.11.21.)
Β§17.11.19 β numRestart=eachSect resets the counter at section boundaries.
Plumbing:
- document-settings.ts:
- readFootnoteNumberRestart / readEndnoteNumberRestart (ST_RestartNumber)
- readSectionNoteConfigs(docPart, w:footnotePr|w:endnotePr) β
Map<sectionIndex, SectionNoteConfig{ numFmt?, numStart?, numRestart? }>
- computeNoteNumbering takes a NumberingOptions struct with sectionConfigs +
defaultRestart + defaultNumFmt. Walks sectionBreak nodes in the PM doc to
track the current section index; resets the counter at section boundaries
when numRestart=eachSect; emits formatById{} keyed by ref id when any
section overrides numFmt.
- ConverterContext: new footnoteFormatById / endnoteFormatById (per-ref
resolved numFmt). Document-wide footnoteNumberFormat remains the fallback.
- inline-converters/footnote-reference + endnote-reference: per-id format
wins over document-wide.
- FootnotesBuilder + EndnotesBuilder: leading-marker formatting honors the
per-id format.
- PresentationEditor: reads document-wide + section-level configs; folds
them into the flow-block cache signature so stale markers invalidate.
Tests:
- document-settings.test.ts: 9 new cases β readers + reader normalization,
Β§17.11.21 pos-ignored case, endnote variant.
- computeNoteNumbering.test.ts: 28 cases total β first-appearance numbering,
customMark suppression, eachSect counter reset (default + per-section
override), per-section numFmt β formatById, backwards-compat (no overrides
β formatById absent).
* feat(footnote): numRestart=eachPage counter math (helper) (SD-2986)
Β§17.11.19 β eachPage restarts numbering at each page boundary.
Page assignment is layout-dependent, so the helper takes an optional
refPageById map populated by a post-layout pass. When present AND the
active restart is 'eachPage', the counter resets when the ref crosses a
page boundary. When absent (first render or non-eachPage docs), the
counter behaves as continuous β gracefully degrading rather than guessing.
Cross-section transition into an eachPage section also triggers a reset
to the next section's numStart (rather than carrying the prior section's
continuous counter), and clears the page tracker so the new section
starts cleanly.
Tests:
- Resets at page boundaries when refPageById is provided.
- Falls back to continuous when refPageById is absent (first-pass shape).
- Section-level eachPage overrides document-wide continuous.
- per-section numStart provides the reset value.
- Cross-section transition (continuous β eachPage) resets cleanly.
Note: the post-layout pass that populates refPageById and re-runs the
layout is intentionally deferred β none of the SD-2986 acceptance docs
uses eachPage and the existing convergence loop already handles
multi-pass without regression. Tracked as a follow-up.
* feat(footnote): classify imported separator + continuationNotice content (SD-2985)
Β§17.11.1 w:continuationSeparator
Β§17.11.23 w:separator
Β§17.18.33 ST_FtnEdn β typed footnote records
Annex L.1.12.5 β continuationNotice text
Foundation for rendering imported separator/continuationSeparator/
continuationNotice content faithfully when the document overrides Word's
default visual (rare in the SD-2985 acceptance corpus, but real for
documents that suppress the separator or specify a pBdr / text).
Two pieces:
1. Importer now preserves continuationNotice typed records (parallel to
separator and continuationSeparator). Empty paragraphs round-trip safely;
explicit content survives in originalXml for the downstream classifier.
2. classifyNoteSeparatorContent inspects the originalXml of a typed record
and returns one of:
- 'default-marker': paragraph contains only <w:r><w:separator/></w:r>
(or continuationSeparator marker). Renderer uses Word's default
visual β Spec A widths already match Β§17.11.1 / Β§17.11.23.
- 'suppression': paragraph is empty. Renderer emits nothing.
- 'explicit': paragraph has w:pBdr (with at least one border defined)
or text content. Consumer converts the XML to FlowBlocks via the
handler chain and emits those fragments instead of the default.
Tests:
- separatorContentClassifier.test.ts (12 cases) β null, empty, marker-only,
pBdr (with + without borders defined), text content, mixed paragraphs,
whitespace-only, continuationSeparator marker.
Visible rendering of the 'explicit' case (toFlowBlocks + layout-bridge
fragment emission) is deferred β none of the SD-2985 acceptance docs uses
non-default separator content, so the implementation is groundwork for
documents in the wild.
* feat(footnote): read + plumb w:pos placement attribute (SD-2986)
Β§17.11.21 w:pos / ST_FtnPos Β§17.18.34 β document-wide footnote placement
attribute, with four enum values: pageBottom (default), beneathText,
sectEnd, docEnd. Per Β§17.11.21 normative text, section-level w:pos is
ignored at render time β only document-wide pos drives behavior.
Foundation:
- readFootnotePosition / readEndnotePosition in document-settings.ts
(rejects unknown values per ST_FtnPos enum).
- ConverterContext gains footnotePosition / endnotePosition fields.
- PresentationEditor reads both up-front and threads them through.
Visible behavior:
- pageBottom (default): unchanged β existing reserve-loop placement.
- beneathText / sectEnd / docEnd: currently fall back to pageBottom
rendering. The reserve-loop fork that places footnote fragments at
the body cursor instead of the page-bottom band is deferred β it's
an architectural change to incrementalLayout.ts that warrants its
own review.
None of the SD-2986 acceptance docs (Simple OnlyOffice, IT-864,
sd-2440) uses non-pageBottom placement, so the literal acceptance
criteria are unaffected by the deferred renderer.
Tests:
- document-settings.test.ts: 4 new cases β all 4 enum values, absent
pos, unknown value rejection, endnote-variant scope.
* fix(footnote): marker is plain superscript + gap before body (SD-2656)
Β§17.11.13 FootnoteRef / Β§17.11.14 footnoteReference β Word's FootnoteReference
rStyle is independent of the first body run's formatting, and Word's source XML
includes a literal space run between <w:footnoteRef/> and the first body run.
Two visible mismatches in `buildMarkerRun`:
1. Marker inherited bold/italic/letterSpacing from the first body text run.
On Keyper Series A the body starts with bold "NTD" β Word renders
"Β³ NTD: ..." (plain marker, bold NTD) but SuperDoc rendered "Β³NTD: ..."
(bold marker, bold NTD, no gap).
2. Marker had no visible separator from body text. Word's source has a
literal space between <w:footnoteRef/> and the first body run; that
space wasn't reaching the rendered output in our pipeline.
Fixes (mirrored in FootnotesBuilder + EndnotesBuilder):
- Drop bold/italic/letterSpacing inheritance from `firstTextRun`. Keep
fontFamily, base size, and color β those are paragraph-level anchors
the marker should share with surrounding context.
- Append `Β ` (NBSP) to the marker text. NBSP survives every
whitespace-collapse path in the line layout, gives a stable gap.
Tests:
- FootnotesBuilder.test.ts: new case asserts marker does NOT inherit
bold/italic/letterSpacing from a bold first text run; existing
expectations updated to "<digit>Β " shape.
Visual verification on Keyper page 6 in dev app:
Before: Β³**NTD**: share classes... (marker bold, no gap)
After: ΒΉ **NTD**: share classes... (marker plain, clear gap)
Refs: SD-2656
* feat(layout-engine): range-aware footnote demand + bodyMaxY-anchored band (SD-2656)
Footnote pagination on the SD-2656 reference fixture matched Word for the
first 18 pages but drifted starting at page 19, ended with 4 extra pages,
and was silently clipping band content past the page bottom on dense pages.
Architectural changes:
- footnoteAnchorsByBlockId now stores per-anchor entries (pmPos + height)
instead of a single block-level total. Demand is queried by range, so
body line-by-line slicing can charge only what the candidate slice
actually anchors β the old "whole-block demand at block entry" charge
over-deferred paragraphs whose first lines anchor few fns but whose
later lines anchor many.
- Body slicer is now range-aware. Each iteration computes the candidate
line's range, looks up its anchored-fn demand + ref count, and adds
that to the page's running total before checking if the line fits.
Pre-slicer advance check previews the first candidate line's demand so
the in-slicer force-commit-first-line rule cannot place a line whose
anchored fn would push the band off the page (the p19 case in the
reference fixture).
- Band painter (incrementalLayout.injectFragments) anchors the band at
page.bodyMaxY instead of pageH - bottomMargin. layoutDocument now stashes
bodyMaxY on each Page after layout settles. This is what Word does β the
separator paints immediately under the last body fragment.
- computeMaxFootnoteReserve uses bodyMaxY when available so the planner's
placementCeiling reflects actual remaining band space. Combined with the
range-aware slicer, fn body that can't fit on its anchor page gets split
into continuation pages instead of overflowing.
- Slicer respects state.pageFootnoteReserve as a floor (alongside
range-aware demand). The convergence loop's reserve communicates
continuation demand from prior pages; without this floor, body packed
the full page on continuation pages and the carried-over fn body
dripped 1 line per page.
- splitRangeAtHeight and fitFootnoteContent no longer charge a range's
spacingAfter when the fitted range completes the input. spacingAfter
is the gap to the next paragraph; for the last item in a band slice
it's wasted budget. The reference fixture's last fn (4 lines Γ 18 px
body + 21 px spacingAfter = 93 px, against an 89-px band budget) was
being force-split to 1 line + 3-line continuation purely because of
this.
Reference fixture results vs origin/main:
- 49 β 46 pages (Word: 45)
- 19/43 β 28/43 footnotes match Word's page exactly
- max drift +4 β +1 page
- 0 band overflows (previously several pages clipped past page bottom)
- last fn body on single page (was splitting across 4 pages)
Corpus-wide layout sweep (`pnpm test:layout --reference 1.32.0`, 562 docs):
- 0 reference / candidate generation failures
- 5 docs with page-count changes β all reductions, none increased
- The 5 are all large legal-template fixtures with many footnotes
- Footnote-only fixtures unchanged page-count
Guard tests:
- New: packages/layout-engine/layout-bridge/test/footnotePageOverflow.test.ts
4 invariants: no fragment past pageH - bottomMargin under clustered fns,
oversized fn body, dense cluster exceeding single band, every ref renders.
- New: packages/layout-engine/layout-bridge/test/footnoteCompleteness.test.ts
Ref-by-ref completeness invariant.
Test status:
- @superdoc/layout-engine: 654/654 pass
- @superdoc/layout-bridge: 1232/1237 pass. The 5 remaining failures test
the legacy fixed-bandTopY + multi-pass-reserve architecture; the
band-at-bodyMaxY model supersedes them. To be retargeted as follow-up.
* chore: remove internal SD-2656 planning docs from branch
Both files are local planning artifacts and should not ship with the PR.
Net effect on main's tree is zero (they were added then removed within
the branch's history).
* fix(footnote): bottom-anchor band painting to match Word convention (SD-2656)
The earlier SD-2656 work painted the band immediately under body
(`bandTopY = bodyMaxY`) to prevent overflow when body packed close to the
band's space. That was correct for the overflow case but inverted Word's
visual convention for the common case: Word anchors the band to the
bottom margin and shows any slack as whitespace BETWEEN body and band;
the prior fix put the whitespace BELOW the band instead.
Per column, compute the total band height from the planner's slice heights
plus separator/divider/padding/gap overhead, then position the band so its
bottom sits at the page's physical bottom margin:
bandTopY = max(bodyMaxY, pageH - originalBottomMargin - totalBandHeight)
- Common case (band shorter than available reserve): the `max` selects
`pageH - bottom - totalBandHeight` β band sits flush against the bottom
margin (Word-style).
- Dense case (band fills its reserve): the `max` selects `bodyMaxY` β
band still hugs body, no overlap. The planner's bodyMaxY-based
`maxReserve` already constrains `totalBandHeight β€ pageBottomLimit -
bodyMaxY`, so the bottom-anchored bandTopY is always β₯ bodyMaxY in
this case.
The original bottom margin is recovered from
`page.margins.bottom - page.footnoteReserved` (the convergence loop
inflates page.margins.bottom by its per-page reserve).
Verified:
- Carlsbad fixture: same 46 pages, identical fn placement, fn 43 still
single page. No regression on the SD-2656 overflow fix.
- Keyper fixture p9 (the visual report case): separator Y now 989 (was
974). Band bottom 1029 β pageBottomLimit 1027. Whitespace shifted
above the band (matches Word convention).
- All 4 footnotePageOverflow guards pass.
- All 2 footnoteBandOverflow guards pass.
- All 3 footnoteCompleteness guards pass.
- @superdoc/layout-engine: 654/654 pass.
* fix(footnote): address PR review comments (SD-2656)
- bodyMaxY: only subtract trailingSpacing when current column's cursorY
owns the page max. Fixes a band-overlap bug in multi-column pages where
column 0 sets maxCursorY high and column 1 ends with non-zero spacing.
- Slicer band overhead now sourced from ctx.getFootnoteBandOverhead,
derived data-driven from topPadding + dividerHeight + separatorSpacingBefore
+ (refs-1)*gap. Planner threads its measured separatorSpacingBefore back
through relayout options so slicer and planner agree on band size.
- computeNoteNumbering: seed counter from numStartFor(0) so section-0
numStart override (Β§17.11.11) applies before the first section boundary.
- eachPage numRestart: coerced to continuous with a one-time warn until the
two-pass pagination handshake exists. Updates the helper doc to flag
refPageById as not wired.
- flow-block cache signature now includes per-id numberById/formatById,
so cached marker text invalidates when ordinals change without a reorder.
- Drop dead slicer state (demandChargedPageNumber, demandLocked,
blockFootnoteDemand) and the unused sliceLines import.
- Add bodyMaxY unit tests (single/multi-column, empty page).
- Direct-string assertions for numberInDash, roman, base-26 letter formatters.
- Retarget footnoteContinuationDemand, footnoteMultiPass, footnoteSeparatorWidth
tests against the bodyMaxY-anchored architecture: bigger body content so
fixtures actually exercise their invariants; drop the multi-pass count
check (now an implementation detail); use page.bodyMaxY as the band-top
anchor instead of pageH - bottomMargin - reserve.
* feat(footnote): split-aware pagination + minimum-start demand model (SD-2656)
Implements Word-like footnote pagination per the SD-2656 plan. The body
paginator now decides line-by-line whether a new fn anchor can stay on
its page based on the MINIMUM first slice of the fn (separator + one
renderable line), not the full body height. The rest of each fn body
splits to continuation pages.
Body slicer (layout-paragraph.ts)
- New ctx.getFootnoteAnchorMinStartForBlockId returns range-aware sum
of measured first-line heights for fns anchored in a PM range.
- computeEffectiveBottom uses minStart for both committed and candidate
demand; state.footnoteDemandThisPage accumulates minStart-only sums
(not full body) so subsequent body blocks on the same page reserve
only the minimum needed for each anchored fn.
Layout-engine planner index (index.ts)
- FootnoteAnchorEntry gains a measured minStart field, defaulted from
options.footnotes.bodyMinStartById or a small height-bounded fallback.
- getFootnoteAnchorMinStartForBlockId exposes the per-range minStart sum
on ParagraphLayoutContext.
Incremental layout bridge (incrementalLayout.ts)
- refreshBodyHeights also builds bodyMinStartById (first paragraph's
first line height, or first-row / first-image-height for non-text
bodies). Threaded through relayout options alongside bodyHeightById.
- placeFootnote forces the first renderable slice of every NEW anchor
(isContinuation=false), not just the first slice on the page. Cluster
pages β many anchored fns on the same body page β now place each fn's
first line regardless of placementCeiling.
- pageReserve propagates the RAW reserve uncapped: capping at maxReserve
stalled convergence when pass-1 body filled the page (maxReserve = 0
-> capped reserve = 0 -> body fills again next pass). Using raw lets
the next pass shrink body to match actual placed band content.
- MAX_FOOTNOTE_LAYOUT_PASSES raised from 4 to 16 to give the monotonic
reserve growth room to settle on dense documents.
- Convergence-loop entry is unconditional when refs exist (pass-1 may
produce zero reserves yet still need iteration).
- findPageIndexForPos now records fallback hits via a module-scoped
tracer (no behavior change) so SD_DEBUG_FOOTNOTES traces surface the
case for diagnostic and test purposes.
- FootnoteLayoutPlan returns structured diagnostics (cappedPages,
pendingFootnoteIds) alongside the existing console.warn behavior so
callers can inspect final-state outcome without parsing logs.
Tracing
- SD_DEBUG_FOOTNOTES env var emits one JSON record per layout pass
describing the final-state anchor->page map, first-slice->page map,
per-page slice ids, reserves, continuation in/out, and any
findPageIndexForPos fallbacks.
- installFootnoteTraceSink(fn) lets tests capture snapshots
programmatically. No-op in production builds.
Tests
- New footnoteIT923Invariants.test.ts pins three Word-fidelity shapes:
page-5 long-fn anchor stays with first slice; page-13 dense cluster
of six anchors all start on the anchor page; page-47 signature-page
anchor stays with its fn body. All three pass.
Results
- IT-923 NVCA fixture: 51 pages -> 46 pages (Word: 49).
- Anchor=firstSlice on every fn ref; no orphan pages; FOURTH on its
page, fn 91 with signature page, exhibit fns 92-94 with EXHIBIT A.
- Body fully used per page (no large whitespace gaps).
- Tests: layout-engine 657, layout-bridge 1240, layout-tests 313,
painter-dom 1100, super-editor footnote subset 93 β all green.
The remaining 3-page deficit vs Word's 49 is canvas-vs-Word text
measurement (paragraphs wrap to fewer lines in Canvas), not a footnote
pagination bug.
* feat(footnote): ordered-cluster rule for anchor placement (SD-2656)
Implements Word's footnote ordered-cluster rule for SuperDoc's
layout engine. For refs [fn1..fnN] introduced on the same body
page, fn1..fnN-1 must render fully on that page; only fnN may
split with overflow flowing forward.
- Track per-anchor firstLineHeight and fullHeight in the
layout-engine state (footnoteAnchorEntries by block id).
- Replace flat-sum demand query with an ordered list
(getFootnoteAnchorsForBlockRange) so the slicer sees the
document-order anchor sequence committed to a page.
- Slicer reservation uses the ordered formula:
required = sum(fullHeight of all-but-last) + firstLineHeight(last)
+ bandOverhead(count).
Adding a new ref upgrades the previous "last" anchor's
contribution from firstLineHeight to fullHeight.
- Planner places ranges via fitFootnoteContent with the
slicer-reserved band height; the cluster math up front
guarantees non-last anchors fit their full body.
- Pageinator carries footnoteAnchorsThisPage (ordered)
alongside footnoteRefsThisPage so the slicer can compose
committed + candidate sequences.
- 4 IT-923-shape invariant fixtures cover p5 (FOURTH), p13
(dense 6-anchor cluster), p47 (signature page), and a
3-anchor fn6/7/8 cluster validating "all-but-last full".
* Revert "feat(footnote): ordered-cluster rule for anchor placement (SD-2656)"
This reverts commit 854a0123228df7852c3a573b69358cb1615d8a40.
* Revert "feat(footnote): split-aware pagination + minimum-start demand model (SD-2656)"
This reverts commit a743c9a7b12e7988291c8cb5d0ca09efab7a2be1.
* feat(footnote): ordered-cluster pagination + caps marker rendering (SD-2656)
Word-fidelity work for footnote pagination on IT-923 NVCA Model COI fixture.
Replaces the per-anchor full-height demand model with Word's ordered-cluster
rule: for a body page with N footnote refs, the first N-1 must render fully
and only the Nth may split. Continuations from prior pages render at the top
of the next page's band (Word's order), with body packing leaving room for
both the carry-forward and the next page's cluster obligation.
## Body slicer + planner (cluster rule)
- contracts/resolved-layout.ts: ResolvedListMarkerItem.run carries allCaps /
smallCaps so the painter can apply text-transform on legal-style list
markers (FIRST/SECOND/THIRD) without the field being stripped at resolve
time.
- layout-engine/src/index.ts: FootnoteAnchorEntry gains firstLineHeight.
getFootnoteAnchorsForBlockId exposes ordered entries; demand helper uses
ordered-cluster formula (sum of full of non-last + firstLine of last).
- layout-engine/src/layout-paragraph.ts: two-mode demand check (preferred
first, ordered as fallback). FootnoteAnchorRef type exported. Pre-slicer
uses preferred-only to push block to next page when cluster can't fit
fully; slicer-loop allows ordered fallback to keep cluster intact when
the last anchor can split.
- layout-engine/src/paginator.ts: PageState.footnoteAnchorsThisPage tracks
the ordered cluster committed to this page.
- layout-bridge/incrementalLayout.ts:
- refreshBodyHeights also computes firstLineHeightById per footnote.
- Planner places continuations FIRST at top of band (Word's order);
cluster room is reserved before continuation placement so a large
inbound continuation cannot starve the new cluster.
- placeFootnote enforces non-last full fit; only the last anchor (or a
continuation) uses forceFirst.
- Per-page reserve carry-forward bumps next page's body reserve by
continuation demand + estimated cluster, capped at the page's physical
capacity.
## Painter: caps mark on level markers
- layout-resolved/src/resolveParagraph.ts: preserve allCaps / smallCaps on
marker.run when reconstructing the resolved item (these were being
dropped, defeating Word's FIRST: SECOND: rendering).
- painters/dom/src/utils/marker-helpers.ts + renderer.ts: apply
text-transform: uppercase when run.allCaps, font-variant: small-caps
when run.smallCaps.
## Numbering: ordinalText / cardinalText
- shared/common/list-numbering/index.ts: add ordinalText (1->First,
2->Second, ..., 100+ falls back to numeric ordinal) and cardinalText
formatters. Without these the NVCA charter's level-1 list rendered as
blank labels.
- shared/common/list-marker-utils.ts: MinimalMarkerRun adds allCaps /
smallCaps fields so they can propagate end-to-end.
## Editor surface
- super-editor presentation-editor/types.ts:
FootnotesLayoutInput.firstLineHeightById threads firstLine heights into
layout for the cluster demand math.
## Tests
- layout-bridge/test/footnoteOrderedCluster.test.ts: invariant cases
(1/2/3-anchor cluster, multi-paragraph non-last footnote). All assert
the rule: non-last completes on anchor page, only last may split.
## Diagnostic toolkit + plan
- docs/architecture/sd-2656-it923-footnote-word-fidelity-plan.md:
empirical baseline, lessons-learned from earlier reverted attempts,
single-PR plan with explicit traps to avoid.
- tools/sd-2656-footnote-analyzer/: read-only diagnostic infrastructure
(capture, diff, align, drift-report scripts) so future regressions on
the rule are quickly auditable. Toolkit produces JSON, markdown, and a
side-by-side HTML report; per-page PNG captures are gitignored.
## IT-923 status
- 47 / 47 SD pages with body anchors satisfy the ordered-cluster rule.
- 94 / 94 footnotes render to completion across the document.
- 11 / 40 Word pages with anchors align exactly; drift trajectory 0 -> +6
over the document, one page per cluster spill.
- Layout-bridge: 1241 tests pass. Layout-engine: 658 pass.
Super-editor: 13192 pass.
* feat(footnote): phase 0 page ledger + invariant diagnostics (SD-2656)
Adds the FootnotePageLedger data structure and per-page tracking. No
behavior change yet; ledger is data-only. Phase 0 is the red/green loop
for the remaining committed-page-planning work.
## Ledger
contracts/src/index.ts: new FootnotePageLedger + FootnoteContinuationEntry
types. Page.footnoteLedger?: FootnotePageLedger.
incrementalLayout.ts:
- FootnoteLayoutPlan now includes ledgersByPage drafts.
- computeFootnoteLayoutPlan captures continuationIn at the start of each
page's processing (before placement consumes pendingForPage), and at the
end records continuationOut from pendingByColumn.
- For each pageSlices snapshot, classifies into mandatorySliceIds,
extendedSliceIds, continuationSliceIds.
- Computes mandatoryReservePx (full of non-last + firstLine of last +
overhead) and actualBandHeightPx (sum of slice heights + overhead).
- injectFragments combines the draft with page.footnoteReserved and stamps
page.footnoteLedger with appliedBodyReservePx and deadReservePx.
## Diagnostics
tools/sd-2656-footnote-analyzer/:
- extract-page-state.js: capture page.footnoteLedger into superdoc-state.json.
- check-ledger-invariants.py: validates four invariants:
I1: actualBandHeightPx <= appliedBodyReservePx (band fits)
I2: mandatorySliceIds covers all anchorIds (rule satisfied)
I3: continuationIn[P] == continuationOut[P-1] (carry parity)
I4: deadReservePx < threshold (default 30 px; drift fuel)
Hard failures on I1-I3; I4 produces warnings.
## What the ledger reveals on IT-923
All hard invariants (I1, I2, I3) hold across all 57 pages.
24 pages have deadReservePx > 30 px. Worst: pages 14, 23, 28, 45, 46, 54
each have 400-600 px of dead reserve. These are the drift fuel for phase 1.
## Doc
docs/architecture/sd-2656-it923-footnote-word-fidelity-plan.md: appended
'Next Phase β Committed Page Planning' section.
## Tests
Layout-bridge: 1241 pass (unchanged). No behavior change in this commit.
* feat(footnote): phase 1 body acceptance uses ordered minimum (SD-2656)
Phase 1 of the committed-page-planning refactor. Body acceptance now
checks ordered demand (full of non-last + firstLine of last) instead of
the preferred / ordered-fallback two-mode it used after the cluster-rule
PR. Body packs tighter against the rule's minimum; the planner can later
use leftover capacity opportunistically (Phase 2).
## Changes
layout-engine/src/layout-paragraph.ts:
- Replace computeDemandsForRange with computeOrderedDemandForRange.
- Pre-slicer effectiveBottom uses ordered demand only β no allowOrderedFallback flag.
- Slicer loop: try ordered, accept if fits, break otherwise. Removed the
preferred attempt that was producing unused reserve.
- sliceDemand commits ordered (was preferred / ordered mixed).
## Ledger diagnostics β tolerance fix
tools/.../check-ledger-invariants.py: I1 (band fits in reserve) now allows
2 px tolerance. Planner uses continuationDividerHeight on the first slice
when isContinuation=true while the ledger overhead uses safeDividerHeight,
which can differ by ~1 px; the tolerance avoids false-positive failures
that aren't real overflows.
## IT-923 impact
- Rule: 44/44 pages still satisfy the ordered-cluster rule.
- Total pages: 56 (down from 57).
- 22 pages still have deadReserve > 30 px, total 6618 px across the doc.
Phase 3 (bounded continuation draining) targets this β it's the
carry-forward bump over-reserving for continuations, not the body slicer.
## Tests
Layout-bridge: 1241 pass (unchanged).
* feat(footnote): phase 3 bounded continuation draining (SD-2656)
Continuations spilled from page P now reserve only the room available
on page P+1 (cluster mandatory takes priority, continuation drains
what's left, capped at the physical band). This is a correctness fix
for the carry-forward bump: prior code could either drop continuations
silently when squeezed out by a new cluster, or overshoot the page's
content area when both demanded more than fit.
## Bump formula
incrementalLayout.ts: continuation carry-forward now computes
overhead = separatorSpacingBefore + dividerHeight + topPadding
nextPageMaxBand = physicalContentHeight - minBodyHeight
clusterRoom = min(nextClusterDemand, nextPageMaxBand - overhead)
continuationRoom = max(0, nextPageMaxBand - overhead - clusterRoom)
continuationToReserve = min(continuationDemand, continuationRoom)
finalReserve = min(nextPageMaxBand, clusterRoom + continuationToReserve + overhead)
The single-overhead-per-band model means cluster and continuation
share one separator block on the continuation page, matching how the
band is actually painted. The min() against nextPageMaxBand prevents
the reserve from exceeding what the next page can physically hold,
which previously could push body content to a negative height when
cluster + continuation collided at the cap edge.
## Tests
- 1241 layout-bridge pass (incl. SD-3050 continuation-aware body
pagination β the test that initially regressed and drove the clamp).
- 658 layout-engine pass. SD-3049 updated to use the anchors getter
instead of the legacy getFootnoteDemandForBlockId, since Phase 1
moved body demand to ordered-cluster from anchors.
- 13192 super-editor pass.
## IT-923 ledger (after phase 3)
Hard invariants I1-I3 hold across all 56 pages (band fits reserve,
every anchor has a mandatory slice, continuationIn/Out parity holds).
Dead-reserve warnings unchanged (22 pages, ~6.6k px total) β phase 3
is correctness, not packing. Dead reserve is phase 4's target.
## Drift trajectory (unchanged from phase 1)
8 events, max +6 pages. 2 remain cluster-spills (phase 2), 6 are
page-break-shifts (phase 4's reserve shrink will close these).
* feat(footnote): phase 4 reserve shrink reclaims dead reserve (SD-2656)
The post-grow tighten loop now reclaims dead reserve on pages where the
planner's current demand is much smaller than what body had reserved on
a prior pass β not just on pages where the planner's demand fell to
zero. This unblocks the convergence loop from staying stuck at an
inflated reserve carry-forward (Math.max-only grow path) when the
continuation chain shrinks across iterations.
## Tighten condition
Previously: tighten only fires when applied >= 8px AND planned === 0
(footnote content shifted off the page entirely).
Now: also fires when applied >= 8px AND applied - planned > 8px,
tightening to `planned` (not 0). The grow loop bumps the reserve back
up if the new bodyMaxY causes plan to demand more after the body
absorbs the freed space. The existing safety net reverts the tighten
if grow can't stabilize or page count increases (cluster spills).
`needsWork` is updated to fire on the same condition so the work-skip
fast path doesn't mask the new opportunity.
## IT-923 ledger after phase 4
pages 56 β 50 (Word: 49)
totalDeadReserve 6692 β 1302 px (80% reduction)
pages > 30px dead 22 β 6
hard invariants I1-I3 all hold
## Anchor drift vs Word (49-page reference)
cumulative drift +6 β +1 pages
aligned pages 11/40 β 14/40
drift trajectory tighter; remaining events are individual Β±1
shifts that cancel rather than accumulating
## Tests
- 1241 layout-bridge pass
- 658 layout-engine pass
- 13192 super-editor pass
* chore(footnote): refresh analyzer diff outputs after phase 4 (SD-2656)
* feat(footnote): preferred-reserve and last-anchor-lines telemetry (SD-2656)
Adds two diagnostic fields to FootnotePageLedger so future Word-fidelity
work can distinguish "mandatory-only" pages (where SD renders only
firstLine of the last anchor) from pages already at Word-like fullness.
No runtime behavior change β pure telemetry plus a new analyzer check
and a marker test for the future page-window scorer.
## New ledger fields
contracts/src/index.ts:
preferredReservePx β Word-like target: full(every anchor) + overhead
lastAnchorRenderedLines β measured lines actually rendered for last anchor
incrementalLayout.ts: the planner computes both during ledger drafting
(preferred sums fullHeight across the page's cluster; lastAnchorRenderedLines
counts ranges actually placed by the planner) and stamps them on
page.footnoteLedger in injectFragments next to mandatoryReservePx and
actualBandHeightPx.
## Analyzer diagnostic
check-ledger-invariants.py: new "mandatory-only" warning fires when
actual_band approx mandatory AND preferred - mandatory > tolerance
AND lastAnchorRenderedLines <= 1
On IT-923 this flags 9 pages (1, 4, 10, 15, 23, 32, 35, 42, 49) where
Word gives the footnote band more vertical space than SD does. Per-page
report adds MandPx / PrefPx / LastL columns.
## Marker test
footnotePreferredReserve.test.ts: 1 active test pins the current
mandatory-fallback baseline so future work doesn't silently regress it.
1 it.skip test documents the desired "single long fn renders >1 line
when room exists" behavior. Will be un-skipped only once the page-
window scorer (follow-up work) can pass it without regressing IT-923
page count or drift.
## Why this lands as telemetry only
Tried switching the body slicer to reserve preferred during this work.
IT-923 regressed: pages 50 -> 54, cumulative drift +1 -> +5, dead-reserve
pages 6 -> 13. The cause is a cascade β pushing body to later pages adds
new clusters there that themselves can't fit preferred, propagating the
reserve inflation. A correct policy needs page-window reasoning (simulate
N pages ahead, accept preferred only when the migration is globally
safe). Tracked as follow-up.
## Tests
- 1242 layout-bridge pass (1 marker test skipped)
- 658 layout-engine pass
* refactor(footnote): clarify preferred reserve scoring
* chore(footnote): keep IT-923 analyzer and plan doc local-only (SD-2656)
Untracks tools/sd-2656-footnote-analyzer/ and
docs/architecture/sd-2656-it923-footnote-word-fidelity-plan.md so the
PR diff no longer includes the local diagnostic toolkit or the working
plan document. The files remain on disk for local use.
To re-introduce them later, decide whether each one should be committed
intentionally (review the contents first) or stay outside the repo via
a local gitignore entry.
* fix(footnote): broaden preferred-reserve candidate filter for partial splits (SD-2656)
Vivienne's feedback on the rendering-fidelity PR called out footnotes
splitting across pages even when Word fits them on a single page.
Repro fixtures: 086 Carlsbad and b89cc7aa.
## Root cause
`isMandatoryOnlyFootnotePage` only flagged a page as a preferred-reserve
trial candidate when:
actual_band β mandatory AND lastAnchorRenderedLines <= 1
The scorer therefore never considered pages where the last anchor rendered
2+ lines and the remainder still spilled. These "partial split" cases are
the most common user-visible bug because the reader has to scroll to the
next page mid-footnote.
Repro on b89cc7aa.docx:
page 16 β anchors=[4], mand=36, pref=82, actual=51, lastL=2, fn4 spilled
Repro on 086 Carlsbad:
page 26 β anchors=[24], mand=42, pref=150, actual=116, lastL=5, fn24 spilled
page 34 β anchors=[36], mand=42, pref=187, actual=61, lastL=2, fn36 spilled
None of these entered the trial set.
## Fix
Adds `isSplitLastAnchorFootnotePage`: a page is also a candidate when its
last anchor appears in continuationOut AND the preferred reserve is
meaningfully bigger than current actual. `getPreferredReserveCandidates`
unions both predicates.
The scorer's accept criteria (no new cluster spills, no new mandatory-only
pages, bounded dead-reserve growth, candidate rendered lines improved)
stays unchanged β only the candidate filter widens.
## Verified
- b89cc7aa.docx: 4 split pages -> 1 split page (Vivienne's screenshot case
on page 16 now renders fn4 fully on the anchor page).
- 086 Carlsbad.docx: 12 split pages unchanged (the remaining cases are
multi-anchor with preferred deltas large enough that the scorer
correctly rejects because of downstream cascade β same global
protection as before).
- IT-923 (NVCA Model COI): 50 pages unchanged. No regression.
- 1253 layout-bridge tests pass (1 new test for the partial-split
predicate, covering Vivienne's b89cc7aa page 16 and Carlsbad page 26
patterns plus a non-spilled counter-example).
- 657 layout-engine, 1136 painter-dom pass.
* fix(footnote): allow extra dead-reserve when trial eliminates a split (SD-2656)
Second iteration on Vivienne's feedback. Previous candidate-filter fix
landed the b89cc7aa page 16 case but page 9 (anchors=[2,3], fn3 spilling)
still split because:
* trial target=130 (full preferred) would eliminate the split
(afterSplit=0, afterLines=1->6) but rejected for dead-reserve-bloat:
148 px doc-wide growth > 128 px threshold;
* trial target=125 then passed globally-safe but didn't fix the split
(afterSplit=1) β the user-visible bug stayed.
The scorer was treating the dead-reserve threshold as absolute. But
eliminating a cluster split is a direct user-visible win that's worth
trading some downstream slack for.
## Fix
In `scoreFootnoteWindow`, double the window and document dead-reserve
allowance when the trial eliminates a cluster split in that scope:
windowAllowance = eliminatesSplitInWindow ? base * 2 : base
docAllowance = eliminatesSplitInDoc ? base * 2 : base
All other accept criteria (page count, new cluster-spills, new
mandatory-only pages, candidate rendered lines improved) stay strict.
Trials that just shift dead reserve without removing a split still hit
the original threshold.
## Verified
- b89cc7aa.docx: 4 split pages -> 0 split pages. Page 9 now renders fn3
fully on the anchor page (actual=130 of preferred=130, lastL=6); page
10 is body-only, matching Word.
- 086 Carlsbad.docx: 12 split pages unchanged. The remaining cases all
reject for `page-count-grew` (bumping reserve pushes body to a new
page) β that's a hard global guarantee unchanged by this fix.
- IT-923: pages 50 unchanged; splits 16 -> 15 (slight improvement).
- 1254 layout-bridge tests pass (1 new test for the relaxation, using
b89cc7aa page 9 ledger values).
* fix(footnote): include continuationIn in mandatory and preferred reserve (SD-2656)
Vivienne flagged Carlsbad pages 22/23 where fn 15 splits with its last
line ("independent of one another.") alone on page 23. Inspection of the
page 22 ledger showed:
anchors=[14, 15], continuationIn=[fn 13, 34px], continuationOut=[fn 15, 34px]
mandatoryReserve=134, preferredReserve=168, actualBand=170
The page actually rendered fn 13 (continuing in from page 21) + fn 14 +
firstLine of fn 15. To render the full fn 15 the band would need
continuation(13) + full(14) + full(15) + overhead β 192 px. But the
ledger's preferredReserve only summed full(14) + full(15) + overhead =
168 px β it didn't account for the unavoidable continuationIn slice.
The scorer's trial ladder is capped at preferredReserve, so it never
tried a target large enough to fit fn 15's tail.
## Fix
In the ledger draft (incrementalLayout.ts), prepend continuationIn's
remainingHeightPx to BOTH mandatoryReserve and preferredReserve, with
the gap between continuation and the anchored cluster. Continuations
from prior pages cannot move anywhere else β they belong in both reserves
as a floor.
## Verified
- Carlsbad page 22 ledger now reports mandatory=170, preferred=205,
exposing the gap to the scorer. (The scorer still rejects the bump
with `page-count-grew` β cascading body migration adds 3 pages
because Carlsbad's body is packed to the brink on every page, a
font-metric symptom that lives below this scorer in measuring-dom.
Out of SD-2656 scope.)
- b89cc7aa: still 0 splits β no regression.
- IT-923: still 50 pages, 15 splits β no regression.
- 1254 layout-bridge tests pass.
* fix(footnote): allow +1 page when trial eliminates a cluster split (SD-2656)
Vivienne flagged Carlsbad page 43 where fn 43 splits across pages 43β44 even
though the full 2-line footnote should fit on page 43 (Word keeps it together
at 45 total pages). Live diagnostics in incrementalLayout + footnote-scorer
showed:
page 42 ledger: preferredReserve=113, actualBand=61, appliedBody=61
trials: 8 attempts (target 113β73), all rejected with `page-count-grew`
because each accepted bump grew pages 45β46
The scorer's binary `after.totalPages > before.totalPages β reject` rule at
footnote-scorer.ts:347-349 refused every trial, leaving the split intact.
Word's apparent behavior here is to grow the document by 1 page to keep a
footnote together when body content is densely packed.
## Variant experiments
Ran 5 variants in the dev server, measured Carlsbad split count per:
V0 baseline 45p / 12 splits
V1 +1 page if eliminates doc-level split 46p / 4 splits β winner
V2 +2 pages 46p / 4 splits (identical)
V3 +3 pages 46p / 4 splits (identical)
V4 unlimited if eliminates split 46p / 4 splits (identical)
V5 V4 + drop hasNewId rotation guard 46p / 4 splits (zero benefit)
V1 captures all available wins. Larger growth caps and dropping the rotation
guard buy nothing measurable β the remaining 4 splits hit different gates
(cluster-spill, new-mandatory-only, dead-reserve-bloat) and need task #144's
page-window scorer to resolve.
## Fix
In footnote-scorer.ts, hoist eliminatesSplitInWindow/eliminatesSplitInDoc
above the page-count check (they already exist 25 lines below) and gate the
rejection:
if (after.totalPages > before.totalPages) {
const grewByOne = after.totalPages === before.totalPages + 1;
if (!(grewByOne && eliminatesSplitInDoc)) return reject('page-count-grew');
}
Reuses the existing diff flag the dead-reserve allowance already computes β
no new types, no new helpers, no safety gates dropped.
## Test updates
Two tests asserted the old V0 behavior (specific page count / split presence)
rather than their genuine invariants. Updated to capture invariants instead:
- footnoteBodyDemand.test.ts: `pages === 3` β `pages <= 4`. The original
"no-recharge" invariant is preserved β anything > 4 would still flag a
per-page-recharge regression.
- footnotePreferredReserve.test.ts: dropped the `continuationOut > 0`
assertion; the genuine invariant ("body anchor stays on page 0") is
unaffected by V1 and still asserted.
## Verified
- Carlsbad: 12 β 4 footnote splits, fn 43 fully fits on page 43.
- layout-engine 657, layout-bridge 1281, painter-dom 1179, super-editor 15770 β all green.
* test(footnote): update parity test import after layout-adapter rename
The footnote-formatter-parity test still imported from the pre-rename
path `@superdoc/pm-adapter/footnote-formatting.js`. Main's refactor
moved this module into super-editor at `@core/layout-adapter`. Updated
the import to use the new alias (configured in vite.sourceResolve.ts)
and refreshed the file's header comment to match.
Verified: @superdoc/layout-tests 332 tests pass.
* fix(footnote): three correctness issues found in code review (SD-2656)
1. Continuation deferral broke source order.
The planner loop iterating pending continuations would push only the
failed entry to nextPending and continue. A later smaller
continuation could then place ahead of the deferred one, rendering
footnotes out of source order. Fix mirrors the anchors-loop pattern:
defer the failed entry plus all later entries and break.
2. Post-reserve relayouts dropped measured separator spacing.
applyReserves called relayout(target) without the planner's measured
separatorSpacingBefore. The body slicer fell back to the 12 px
default while the planner sized the band with the measured value,
so body packed too much and the band painted past its budget.
3. advanceColumn carried per-page footnote counters into the next column.
Footnotes are reserved per-column in the planner; the body slicer's
ordered-cluster demand formula must reset per-column or column N
over-reserves for column N-1's footnotes. Fix resets the per-column
counters on column advance. Field names retain "ThisPage" for
back-compat.
## Verified
- layout-bridge 1281, layout-engine 657, layout-tests 332 β all green.
- Carlsbad: 46p / 4 splits β 46p / 3 splits (fn 38 absorbed).
- IRA: 45p / 13 splits β 45p / 17 splits (correctness exposure β the
buggy column-state carryover was masking 4 splits by over-reserving
column 2; the splits were always present, now visible).
* feat(footnote): absorb one-line footnote widows by bumping reserve (SD-2656)
Adds a `runWidowOrphanAbsorb` pass between the convergence loop and the
preferred-reserve scorer. For every page whose predicted footnote tail
is one line short (β€ 24 px), bumps the reserve to the page's preferred
value, bypassing the scorer's page-count-growth gate.
The scorer's gate exists to prevent global regressions when a trial
trades local fidelity for added pages. For one-line widows the trade
is bounded β Word's pagination always absorbs them. The implementation
reuses the existing buildFootnoteLedgers, applyReserves, growReserves,
and capReserveForRelayout helpers; the only new logic is the threshold
filter and the unconditional bump.
## Threshold rationale
Threshold = 24 px (one line of footnote text plus slack). Measurements
on the Carlsbad fixture: at threshold = 35 px the absorb pass creates
new cluster splits on pages 25-29; at threshold = 24 px no regression
is measurable. 24 is the largest value with a clean profile across the
two test fixtures.
## Trade-off
This pass may grow the document to absorb widows. On the IRA fixture,
six one-line widows bump cleanly but force the doc 45 β 48 pages. The
"revert on grow" guard would make the pass a no-op everywhere unless a
doc has body slack (test fixtures do not). The trade is accepted for
docs whose layouts genuinely have nowhere to absorb a widow without
growth. Future work pairs this with body paragraph widow/orphan
controls so the body absorbs the pushed line for free.
## Verified
- layout-bridge 1281, layout-engine 657, layout-tests 332 β all green.
- Carlsbad: unchanged at 46p / 3 splits (no one-line tails to absorb).
- IRA: 45p / 17 splits β 48p / 9 splits (8 widows absorbed, 3 page cost).
* feat(footnote): reserve full footnote demand at body slice time (SD-2656) (#3597)
Replaces the body slicer's ORDERED-MINIMUM acceptance rule with
ORDERED-PREFERRED. The slicer now reserves each anchored footnote's
full height up front, instead of just the first line of the last
anchor. The body naturally backs off enough lines to fit every
anchored footnote whole on its anchor page β matching Word's
pagination behavior, which knows each footnote's full demand at
every line decision rather than reserving a minimum and patching
later.
## Architectural rationale
The previous five-layer pipeline (mandatory-minimum planner β body
slicer β convergence loop β preferred-reserve scorer β post-hoc widow
absorb) existed to compensate for the deliberate under-reservation
at layer 1. Each downstream layer fixed a symptom of layer 1's
optimism. By reserving the full demand at slice time, the symptoms
disappear and the downstream layers can be simplified or removed in
follow-up work.
This is the cleaner shape: one place that decides demand, no
back-and-forth between layers.
## Fixture results
| Fixture | Before | After |
|---|---|---|
| Carlsbad | 46p / 3 splits | 46p / 0 splits |
| IRA | 48p / 9 splits | 46p / 0 splits |
| SPA | 53p / 7 splits | 53p / 0 splits |
| IT-923 COI | 50p / 15 splits (Phase 1 era) | 54p / 1 split |
| MRL | 5p / 0 splits | 5p / 0 splits |
Cost is a small page-count growth (β€ +4 pages on packed legal docs
like COI; β€ +1 on most others). Word would also grow these documents
under similar packing pressure.
The single remaining split (COI fn 32) is a footnote large enough
that no single page accommodates it without itself overflowing β a
genuine forced split that Word would also produce.
## Test sweep (all green)
- layout-engine 657 / layout-bridge 1281 / layout-tests 332
The Phase 1 dead-reserve concern (24 IT-923 pages had `deadReserve >
30 px` under preferred demand) is mitigated by the codex correctness
fixes shipped earlier on the SD-2656 branch β the column-state
carryover that exaggerated dead-reserve drift is gone.
|
π Docs preview: https://superdoc-merge-main-into-stable-2026-06-02.mintlify.app |
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a594ce2e68
βΉοΈ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (typeName === 'sectionBreak') { | ||
| const nextSection = sectionIndex + 1; | ||
| // Β§17.11.19 β at section boundary, reset the counter to the next section's numStart | ||
| // when its restart policy is anything other than continuous. (For continuous, the counter | ||
| // carries through from the previous section.) Also clears the page tracker so eachPage | ||
| // logic restarts cleanly inside the new section. | ||
| const nextRestart = restartFor(nextSection); | ||
| if (nextRestart === 'eachSect' || nextRestart === 'eachPage') { | ||
| counter = numStartFor(nextSection); |
There was a problem hiding this comment.
Derive note sections from real section metadata
For imported DOCX files with section-level w:footnotePr/w:endnotePr after the first section, this never advances because the PM document does not carry the layout-adapter sectionBreak FlowBlocks being checked here; those are emitted later from section ranges in layout-adapter/internal.ts before toFlowBlocks handles content. As a result, refs in later imported sections keep using section 0/default numbering and formats, so numStart, numFmt, and eachSect overrides parsed by readSectionNoteConfigs() are silently ignored for the common DOCX path.
Useful? React with πΒ / π.
| const colorWithAlpha = (color: string, alpha: number): string => { | ||
| const expanded = color.trim().startsWith('#') ? expandHexColor(color.trim()) : null; | ||
| if (!expanded) return color; |
There was a problem hiding this comment.
Avoid using opaque CSS colors as highlight backgrounds
When a host resolver returns a valid non-hex CSS color such as rgb(31, 111, 235) or red (the public API accepts CSS color strings), colorWithAlpha() returns that value unchanged and the caller assigns it to --sd-tracked-changes-*-background. That makes insert/delete highlights render as a fully opaque author color instead of a translucent tint, obscuring text in review mode; non-hex colors should either be converted to an alpha-capable form or leave the background variable at the default.
Useful? React with πΒ / π.
Codecov Reportβ All modified and coverable lines are covered by tests. π’ Thoughts on this report? Let us know! |
Summary
merge/main-into-stable-2026-06-02fromstablemaininto the candidate branchstableAuto-created by promote-stable workflow.